1 link tagged with all of: jev + healthcare + evaluation + zero shot
Click any tag below to further narrow down your results
Links
@muratcan on X
The post reports an experiment where Jev, an AI model, analyzed 2,029 real phone calls handled by an AI receptionist, using only call structure (turns, tool calls, workflow stages, timing) and never the audio or transcripts. Jev predicted which calls would end in a booking with an AUC of 0.78 at the halfway point, rising to 94% correct ranking near the end, at a cost of about $3.
- Jev never heard the calls; it worked only from structural signals like turns, tool calls, and workflow stages.
- It made 38,012 turn-level forecasts at a 118 ms median latency and reviewed all calls in 26 seconds.
- Zero-shot, with no fine-tuning, it reached an AUC of 0.78 mid-run and ranked calls correctly 94% of the time by the end, though it overweighted visible errors the agent usually recovers from.